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Abstract 

Many publications and surveys refer to the high drop out rate in Massive Open Online Courses (MOOCs) 
which is around 90%, especially if we compare the number of students who register against those who finish. 
Working towards improving student engagement in MOOCs, we focus on providing specific research-based 
recommendations on formative assessment and feedback practices that can advance student activity. In this 
respect, we analysed some significant research papers on formative assessment and feedback methods 
applicable to face-to-face teaching environments that advance student engagement, and concluded with 
related requirements and conditions that can be applied also to MOOCs. We also analysed 4050 comments 
and reviews of the seven most active and highly rated MOOCs (6 Coursera ones and 1 from EdX) provided 
by the students who have mainly completed those courses via CourseTalk. Based on this content analysis, 
we have formulated fourteen recommendations that support also the requirements/conditions of our conceptual 
and theoretical framework analysis. 

The results obtained shed some light in a rather unexplored research area, which is the research on 
formative assessment and feedback practices specifically for stronger engagement in MOOCs. 

Keywords: MOOCs, formative assessment, feedback, student engagement, peer assessment, self- 
assessment 


Introduction 

Massive Open Online Courses are the latest trend in the area of e-learning and remote education 
and have significant popularity in the higher education community. For instance, as of July 2014 
more than 8.2 million students have enrolled in at least one course in Coursera and more than 678 
courses have been offered by 110 Universities and other institutions (Anderson, 2014). However, 
the drop out rates are very high in the range of 90% or more (Clow, 2013; Lewin, 2013) and the 
research community should focus on addressing this issue by trying to understand the causes and 
suggest specific solutions so that open education achieves its high potential and does not fail. In 
this respect, there is already some research activity on identifying the factors that influence student 
engagement that can be categorized into two broad categories: a) the non-didactic ones (students’ 
and instructors’ profiles, their demographics, reputation of institutions and of the teaching staff 
involved, certification options, fee options, course popularity, etc.) and didactic ones such as course 
structure and content, self-paced or not, workload and duration, course topic, type of exams, type 
of assessments and feedback, and interaction with students and instructors, etc. (Adamopoulos, 
2013). Nevertheless, little research has been carried out to establish whether different types of 
assessment (formative, non-formative, peer and self-assessment) or different ways to provide 
feedback to students may actually affect course quality and impact (negatively or positively) on the 
MOOC students’ learning experience and consequently on their engagement with the MOOCs. In 
this respect, we have built a conceptual and theoretical framework of requirements related to 
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feedback and formative assessment and based on this framework, we have analysed the reviews 
provided by students on related feedback and formative assessment practices applied in popular 
and highly ranked MOOCs. 

The objective of this study is twofold: 

• Objective 1: To formulate a framework of requirements and conditions for effective feedback 
and formative assessment practices in MOOCs based on literature. 

• Objective 2: To suggest specific recommendations on feedback and formative assessment 
practices applied in MOOCs for increasing student engagement. 

Our first objective is addressed through the conceptual and theoretical framework analysis (following 
section) and the second one is achieved through the analysis of the selected samples and the 
related content (methodology and results). 

Conceptual and Theoretical Framework Analysis 

In this paper we focus specifically on formative assessment and feedback practices that could be 
applied in MOOCs for enhancing student engagement. 

The proposed paper is based on the identification and the analysis of specific research and 
scientific papers that conclude with recommendations on formative assessment and feedback that 
can be applied especially in MOOCs to advance student engagement (i.e. increasing students’ 
activity) and acquisition of knowledge. 

Although there is no evidence yet on which specific assessment and formative models can be 
applied in MOOCs for advancing student engagement, the research carried out by Nicol and 
Macfarlane-Dick (2006) was analysed further to examine whether its recommendations, if applied 
also in fully online eLearning environments including MOOCs, can support student engagement. 

Nicol and Macfarlane-Dick (2006) argue that there are three conditions as pre-requisites for 
students to be familiar with in order to benefit from feedback in academic tasks, i.e. students should 
be familiar in advance with: 

1. what good performance is (i.e. the student must possess a concept of the goal or standard 
being aimed for); 

2. how current performance relates to good performance (the student must be able to compare 
current and good performance); 

3. how to act to close the gap between current and good performance. 

The above three pre-requisites imply that in order for students to be able to compare an actual 
performance (of their own or of their peers) with a standard good one and take action to close the 
gap, the whole training process should dedicate much more effort on strengthening the students’ 
self-assessment skills for better learning experiences. In this context, Nicol and Macfarlane-Dick 
(2006) have identified seven principles for good formative assessment and feedback that should 
be applied in traditional teaching environments in order to strengthen the students’ capacity to self- 
regulate their own performance. These seven principles, in brief, suggest that effective assessment 
and feedback practices should a) help clarify what good performance is, b) facilitate the development 
of self-assessment (reflection) in learning, c) deliver high quality information to students about their 
learning, d) encourage teacher and peer-dialogue around learning, e) encourage positive motivational 
beliefs and self-esteem, f) provide opportunities to close the gap between current and desired 
performance, and g) provide information to teachers that can be used to structure the teaching 
approach. 
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Similarly, other research (Gibbs & Simpson, 2004) argues that assessment has positive effect on 
students’ learning and engagement and proposes a set of conditions for this to happen in traditional 
teaching environments. In order for student engagement to be enhanced, assessment tasks should 
a) be sufficient to require students to dedicate appropriate study time, b) orientate students to 
allocate appropriate amounts of time and effort to the most important aspects of the course, c) 
engage students in related productive learning activities. Furthermore, feedback should a) be 
provided in sufficient detail and often enough, b) focus on students’ performance, on their learning 
and on actions under the students’ control, rather than on the students themselves and on their 
characteristics, c) be delivered in time for students so that it still matters for them, d) be aligned 
with the purpose of the related assignment e) be well received by the student and f) advance future 
learning and use by the student. 

Additionally, according to Hew (2015), student engagement in MOOCs is defined as the level of 
a student’s engagement in a learning activity. The more the student is active within a course, the 
more engaged she/he is with this course. Furthermore, Hew reviews specific literature (Fredricks, 
Blumenfeld & Paris, 2004; Helme & Clarke, 1998) on student engagement and has identified its 
three main dimensions: 

1. behavioral engagement referring to the learning activities that students are doing within a 
course such as completing an assignment, watching videos, participating in forums, etc, 

2. affective engagement referring to the feelings that learning activities generate in students 
towards other colleagues, tutors, the course itself or the institution that runs the course 

3. cognitive engagement referring to the emerging thoughts that learning activities provoke in 
students, e.g. cognition activity for asking and answering questions, for giving clarifications, 
for reasoning, etc. 

Furthermore, Hew with the support of other literature (Reeve, 2012; Skinner, Kindermann, Connell 
& Wellborn, 2009) directly links student engagement with motivation and more specifically with Self- 
Determination Theory (STD) (Deci & Ryan, 1991; Deci & Ryan, 2000; Hardre and Reeve, 2003) 
and concludes that student engagement and more specifically 

1. behavioral engagement is driven from the need of autonomy (the need students feel to sense 
they are non dependent on other peoples’ actions), 

2. affective engagement is driven from the need of relatedness (the need students feel to 
connect with other people) and 

3. cognitive engagement is driven from the need of competence (the need students feel to 
master specific knowledge). 

Based on this conceptual and theoretical analysis, we can argue that formative feedback and related 
assessment methods can support student engagement as long as they follow a specific framework 
of requirements and conditions: 

1 . Requirement 1 : The course should fulfill the need of autonomy and consequently behavioral 
engagement that is for example addressed through self-assessment practices (i.e. a student 
to assess their own work and assignments), 

2. Requirement 2: The course should fulfill the need of relatedness and consequently affective 
engagement that is for example achieved through peer assessment practices (i.e. a student 
to assess the work and assignment of other students/peers) 

3. Requirement 3: The course should fulfill the need of competence and in this manner also 
cognitive engagement through for example formative assessment and feedback (that is, 
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assessment specifically intended to generate feedback on performance and improve and 
accelerate learning) 

4. Requirement 4: Students should know in advance what good performance is and based on 
that to be able to compare assessments 

5. Requirement 5: Students should know in advance the necessary actions needed to reach 
good performance 

6. Requirement 6: Formative assessment and feedback practices should help students clarify 
what good performance is and how much different this is with respect to their current 
performance 

7. Requirement 7: Formative assessment and feedback practices should encourage dialogue 
around learning between peers and teachers 

8. Requirement 8: Formative assessment and feedback practices should create positive 
motivational beliefs and self-esteem 

9. Requirement 9: Formative assessment and feedback practices should encourage students 
to take actions in order to achieve good performance 

10. Requirement 10: Formative assessment and feedback practices should provide information 
to teachers to improve their teaching approach 

11. Requirement 11: Assessment tasks should be sufficient in order to require students to 
dedicate appropriate study time for addressing them 

12. Requirement 12: Assessment tasks should orientate students to allocate appropriate amounts 
of time and effort to the most important aspects of the course 

13. Requirement 13: Assessment tasks should engage students with related productive learning 
activities 

14. Requirement 14: Feedback should be provided in sufficient detail and frequently enough 

15. Requirement 15: Feedback should focus on students’ performance regarding their learning 

16. Requirement 16: Feedback should be delivered in time to students so that it still matters and 
it is well received 

17. Requirement 17: Feedback should be aligned with the purpose of the related assignment 
and advance future learning and use by the student 

The above analysis shapes the necessary framework of requirements to analyse students’ reviews 
on highly rated MOOCs and to formulate related recommendations on formative assessment and 
feedback methods that can advance student engagement. 

Methodology 
Selection Process/Sample 

In our approach, we have used a special online review platform for MOOCS, i.e. CourseTalk 
(www.coursetalk.com) and we have applied the following content analysis methodology. 

1. We have collected all the responses from all the e-courses reviewed at Coursetalk that satisfy 
all of the following conditions based on the data available on Coursetalk on 24 th November 
2014: 

• they are offered for free and 

• they are offered by Universities and 

• they are top-rated, i.e. 5/5 stars and 

• they have received more than 100 reviews (in order to have a significant content to analyse) 
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2. Based on the above, we have identified the following 7 MOOCs and around 4050 reviews for 
our content analysis 

• An introduction to Interactive Programming in Python, offered by Coursera (1863 reviews) 

• A beginner’s Guide to Irrational Behavior, offered by Coursera (375 Reviews) 

• Epidemics - the Dynamics of Infectious Diseases offered by Coursera (315 Reviews) 

• Design: Creation of Artifacts in Society, offered by Coursera (204 Reviews) 

• Modern and Contemporary American Poetry, offered by Coursera (170 Reviews) 

• An introduction to Operations Management, offered by Coursera (104 Reviews) 

• ThinklOIx: The Science of Everyday Thinking, offered by EdX (1037 Reviews) 

The main formative assessment and feedback methods that have been applied in all those courses 
have focused on peer-assessments and feedback through discussion forums with the participation 
of the instructors in some cases or of their assistants as well. 

Content Analysis Process Steps 

The total 4050 reviews include all the comments/reviews received by all participants that have 
mainly completed any of the seven MOOCs. Therefore, in order to analyse these reviews and 
conclude with specific recommendations related to feedback and formative assessment practices 
that advance student engagement, we have applied five specific steps: 

• Step 1: filtering and selection of comments related to feedback and formative assessment 

• Step 2: further selection of those comments that specifically contribute or provide some input 
on the recommendations to be formulated 

• Step 3: Reading each one of the finally selected comments and formulation of specific 
recommendations based also on the requirements from the analysis of the conceptual and 
theoretical framework. Special attention is given so that total suggested recommendations 
are linked with the requirements that were identified in our conceptual and theoretical 
framework analysis in a previous section in this paper. 

• Step 4: Re-reading each one of the finally selected comments and identification of a) how 
many times each review/comment was used as source for each recommendation and 
b) whether there were any other recommendations raised that may have been missed from 
the previous step or whether there were any further revisions needed. Our content analysis 
took place mainly at step 3 and step 4 and in order to validate its findings, two additional 
judges that had not been initially involved in the development process of steps 3 and 4 were 
engaged. They applied steps 3 and 4 in a sample of reviews in order to ascertain agreement 
on the findings. 

• Step 5: Based on the number of reviews/comments that are used as sources for each of the 
formulated recommendations, we identify the significance of each recommendation and we 
rank them accordingly. 

Step by step Approach - Results 

Below, we outline each step in our approach and their results. 

Step 1 

Out of the 4050 reviews, we collected all responses that specifically commented on feedback and 
assessment methods by identifying them with the search terms “Feedback”, “Assessment”, “Peer” 
and “Assignment”. In total, we have collected 418 reviews. 
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Step 2 

Out of the 418 reviews, we eliminated those that were general, either positive or negative, and were 
not contributing or providing any specific details in regards to feedback and formative assessment 
practices that have been applied in each MOOC and we were left with 237 comments/reviews in 
total. 

Step 3 

We carefully read each of the 237 comments/reviews and we used each one of them as a reference 
source to elaborate a specific recommendation on feedback and formative assessment for advancing 
student engagement. Each review was checked carefully on whether it was the source of a new 
recommendation or the source of an already raised recommendation from previous comments/ 
reviews. Also each of the formulated recommendations was checked concerning whether it was 
related to any of the sixteen requirements from our conceptual analysis. Based on this approach, 
we ended up with a total of fourteen recommendations/findings which are all linked to our main 
requirements on feedback and formative assessment practices for higher student engagement as 
shown in table 1. 


Table 1: Identified Recommendations on feedback and formative assessment practices for 
advancing student engagement and their links to related requirements 


Recommendations/Findings 

Main Related 
Requirements from 
Section 2 

Recommendation/finding 1: Assessment models, such as self assessments 
(students judge their own work) and peer-assessments (students judge the 
work of their peers) can be relatively complex in MOOCs and a comprehensive 
rubric should be provided in all MOOCS that involve peer assessments. 

Requirements 4, 5, 6, 

9, 12 

Recommendation/finding 2: Self and peer assessments can be obligatory or 
penalties can be applied and should have as priority that peers examine the 
work of others and provide constructive feedback (furthermore, if possible, 
students should be able to modify their own related assignment work even after 
its submission based on feedback received). 

Requirements 4, 6, 9, 

14, 15 

Recommendation/finding 3: Although MOOCs can automatically assess 
quizzes, self and peer assessments must deal with more comprehensive 
assignments. However, self- and peer - assessment should be formative as 
well, i.e. specifically intended to generate feedback on performance and 
improve and accelerate learning rather than just providing a mark. 

Requirements 3, 6, 8, 

9, 11, 15, 

Recommendation/finding 4: Peer assessment in MOOCs does not always 
deliver accurate results and in many cases it creates frustrations or negative 
feelings since it doesn’t provide any feedback on the work assessed or there 
are concerns on the competencies of the peer-assessor. 

Requirements 2, 3, 4, 

5, 6, 8,9, 14, 15 

Recommendation/finding 5: Peer assessment was more welcome in those 
MOOCs where there was no mark given or alternatively feedback was given on 
the quality of the related assignment. 

Requirements 1, 2, 3, 

4, 5, 6, 8, 9, 14, 15, 
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Recommendations/Findings 

Main Related 


Requirements from 
Section 2 

Recommendation/finding 6: Immediate feedback mainly from automated 
simple but well structured quizzes is appreciated by students since it confirms 
their understanding about what they have learnt. The appreciation is even 
higher in cases in which some further feedback on the given answer or on the 
correct answer is provided. 

Requirements 1, 3, 6, 

9, 14, 15, 16 

Recommendation/finding 7: No limitation on number of attempts in quizzes is 
appreciated by students and actually helps them to understand the material 
more thoroughly. 

Requirements 1, 3, 6, 

9, 16 

Recommendation/finding 8: Feedback on assignments can be provided in 
time before the next assignment so that students are able to use the 
suggestions provided in the next one. 

Requirements 1, 3, 9, 

16 

Recommendation/finding 9: Peer assessment can be supplemented with 
related forum discussions for interaction and the possibility to offer a 
communication channel for clarifications. 

Requirements 2, 3, 6, 

7, 9, 14, 15, 16 

Recommendation/finding 10: A well structured course syllabus with 
information on what assignments are required per week/module by each 
participant, by when, and its related training content was appreciated. 

Requirements 1, 9, 11, 
12, 

Recommendation/finding 11: All assignments in one course or module that 
are assessed by peers could be always allocated to the same peers instead of 
being randomly allocated, in order to enable follow up on the progress of each 
peer within the same group. 

Requirements 2, 6 

Recommendation/finding 12: The posting of comments by human experts, for 
instance instructors’ assistants that intervene and provide comments and views 
in discussion forums or even in students’ work is appreciated. 

Requirements 1, 2, 3, 

6, 7, 9, 10, 14, 15, 16 

Recommendation/finding 13: Discussion forums are appreciated and are a 
very good way for students to receive support, sympathy, formative feedback 
and clarification as well as to share ideas on their work as long as there is an 
effective mechanism on managing and accessing the discussion threads. 

Requirements 1, 2, 3, 
4,5, 6, 7, 8, 9, 10, 14, 

15, 16 

Recommendation/finding 14: Assignments can be based on practical 
problems with clear expected output that give the sense of completeness to the 
students and are not that easy to address but are sufficiently challenging and 
interesting. 

Requirements 1, 2, 3, 

9, 12,13 


Step 4 

Then, we read again each of the 237 comments/reviews in order to a) identify how many times 
each review/comment was used as source for each recommendation and b) check whether there 
were any other recommendations raised that may have been missed from the previous step or 
whether there were any revisions needed on the 14 recommendations. Consequently, we haven’t 
identified any other recommendations apart from the 14 initial ones. Also the text of three of them 
was slightly revised, which is addressed above. Based on this step, table 2 was prepared, showing 
for each course: the total number of reviews given (Column 3), the number of potentially useful 
reviews (Column 4), the number of identified useful reviews (Column 5). In the same manner, table 
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3 shows the number of reviews used as sources for each recommendation (Columns 2- Column 
15). Also the overall totals are shown accordingly. In addition, in order to avoid having the courses 
with a large number of related reviews to influence the final ranking of the recommendations, we 
have applied the weighted average method for each recommendation as is shown in the next step. 


Table 2: Number of reviews per specific category and per MOOCs 


MOOC Titles 

MOOC 

Platform 

Total 

Reviews 

Comments based on 
search terms 
“Feedback”, 
“Assessment”, “Peer”, 
“Assignment” 

Useful reviews 
specifically on 
feedback and 
formative 
assessment 

Course 1: An introduction to 
Interactive Programming in 

Python 

Coursera 

1863 

307 

174 

Course 2: A beginner’s Guide to 
Irrational Behavior 

Coursera 

375 

12 

7 

Course 3: Epidemics - the 
Dynamics of Infectious Diseases 

Coursera 

315 

6 

4 

Course 4: Design - Creation of 
Artifacts in Society 

Coursera 

204 

32 

17 

Course 5: Modern and 
Contemporary American Poetry 

Coursera 

170 

23 

16 

Course 6: An introduction to 
Operations Management 

Coursera 

104 

22 

11 

Course 7: ThinklOIx: The 

Science of Everyday Thinking 

EdX 

1037 

16 

8 

Total Instances 


4068 

418 

237 


Step 5 

We applied the weighted average method for each recommendation. For example, for Recommendation 

1, the weighted average is (17x174+3x7+0x4+7x17+3x16+0x11+1x8)/237=13 

We have calculated the weighted average for each of the 14 recommendations as is shown on 
table 3 above based on the number of reviews that were used as sources for each of the fourteen 
recommendations. In this way, we have avoided the influence in the ranking of one course with 
many comments. Based on the weighted average derived for each recommendation, we have 
ranked them accordingly as the following chart shows (see Chart 1). 

Therefore, our recommendations/ findings are ranked and provided below from the highest to the 
lowest significance by considering also their weighted average. 

a. Recommendation/finding 14: Assignments can be based on practical problems with clear 
expected output that give the sense of completeness to the students, and are not that 
easy to address but are sufficiently challenging and interesting (65 reviews supported this 
recommendation/finding with 42 as weighted average) 
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Table 3: Number of reviews per specific recommendation and the related weighted average 


MOOCS 

No. of 

reviews 

for 

Recom. 

1 

No. of 

reviews 

for 

Recom. 

2 

No. of 

reviews 

for 

Recom. 

3 

No. of 

reviews 

for 

Recom. 

4 

No. of 

reviews 

for 

Recom. 

5 

No. of 

reviews 

for 

Recom. 

6 

No. of 

reviews 

for 

Recom. 

7 

No. of 

reviews 

for 

Recom. 

8 

No. of 

reviews 

for 

Recom. 

9 

No. of 

reviews 

for 

Recom. 

10 

No. of 

reviews 

or 

Recom. 

11 

No. of 

reviews 

for 

Recom. 

12 

No. of 

reviews 

for 

Recom. 

13 

No. of 

reviews 

for 

Recom. 

14 

Course 1 

17 

4 

41 

12 

13 

10 

0 

0 

4 

28 

0 

4 

24 

57 

Course 2 

3 

0 

3 

4 

4 

0 

0 

0 

0 

1 

0 

0 

0 

0 

Course 3 

0 

0 

2 

1 

0 

0 

0 

0 

0 

0 

0 

0 

1 

2 

Course 4 

5 

0 

2 

6 

5 

0 

0 

2 

2 

1 

2 

1 

1 

2 

Course 5 

3 

0 

0 

0 

1 

1 

0 

0 

3 

3 

0 

5 

9 

0 

Course 6 

0 

0 

0 

1 

0 

2 

2 

0 

0 

2 

0 

1 

0 

4 

Course 7 

1 

0 

0 

2 

0 

5 

0 

0 

0 

0 

0 

0 

1 

0 

Total 

Instances 

29 

4 

48 

26 

23 

18 

2 

2 

9 

35 

2 

11 

36 

65 

Weighted 

Average 

13 

3 

30 

9 

10 

8 

0 

0 

3 

21 

0 

3 

18 

42 


Significance of each recommendation based on no. of linked reviews 


Recommendaton 14 


Recommendation 3 


Reconrmepdaton 10 


Recommerdaton 13 


Recommendation 1 


Recommendation 5 


Recommendation 4 


Recommendation 6 


Recommendaton 12 


Recommendation 9 


Recommendation 2 


Recommendation 11 


Recommendation 8 


Recommendation 7 



0 S 10 15 20 25 30 35 40 45 


Chart 1: Ranking of each recommendation with respect to its significance/no. of linked reviews 
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b. Recommendation/finding 3: Although MOOCs can automatically assess quizzes, self and 
peer assessments must deal with more comprehensive assignments. However, self- and peer 
assessment should be formative as well, i.e. specifically intended to generate feedback on 
performance and improve and accelerate learning rather than just providing a mark. (48 
reviews supported this recommendation/finding with 30 as weighted average) 

c. Recommendation/finding 10: A well structured course syllabus with information on what 
assignments are required per week/module by each participant, by when, as well as its related 
training content was appreciated (35 reviews supported this recommendation/finding with 21 
as weighted average) 

d. Recommendation/finding 13: Discussion forums are appreciated and are a very good way 
for students to receive support, sympathy, formative feedback and clarification as well as 
share ideas on their work as long as there is an effective mechanism on managing and 
accessing the discussion threads. (36 reviews supported this recommendation/finding with 
18 as weighted average) 

e. Recommendation/finding 1: Assessment models, such as self assessments (students 
judge their own work) and peer-assessments (students judge the work of their peers) can be 
relatively complex in MOOCs and a comprehensive rubric should be provided in all MOOCS 
that involve peer assessments (29 reviews supported this recommendation/finding with 13 as 
weighted average) 

f. Recommendation/finding 5: Peer assessment was more welcome in those MOOCs where 
there was no mark given or alternatively feedback was given on the quality of the related 
assignment. (23 reviews supported this recommendation/finding with 10 as weighted average) 

g. Recommendation/finding 4: Peer assessment in MOOCs does not always deliver accurate 
results and in many cases it creates frustrations or negative feelings since it doesn’t provide 
any feedback on the work assessed or there are concerns on the competencies of the peer- 
assessor. (26 reviews supported this recommendation/finding with 9 as weighted average) 

h. Recommendation/finding 6: Immediate feedback mainly from automated simple but well 
structured quizzes is appreciated by students since it confirms their understanding about 
what they have learnt. The appreciation is even higher in cases that some further feedback 
on the given answer or on the correct answer is provided. (18 reviews supported this 
recommendation/finding with 8 as weighted average) 

i. Recommendation/finding 12: The posting of comments by human experts, for instance 
instructors’ assistants that intervene and provide comments and views in discussion forums 
or even in students’ work is appreciated. (11 reviews supported this recommendation/finding 
with 3 as weighted average) 

j. Recommendation/finding 9: Peer assessment can be supplemented with related forum 
discussions for interaction and the possibility to offer a communication channel for clarifica¬ 
tions. (9 reviews supported this recommendation/finding with 3 as weighted average) 

k. Recommendation/finding 2: Self and peer assessments can be obligatory or penalties can 
be applied and should have as priority for the peers to examine the work of others and provide 
constructive feedback (furthermore, if possible, students should be able to modify their own 
related assignment work even after its submission based on feedback received). (4 reviews 
supported this recommendation with 3 as weighted average) 

l. Recommendations 11, 8 and 7 are not considered since they have weighted average close 
to 0. 
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Conclusions 

We have analysed existing significant research papers that deal with formative assessment and 
feedback methods as well as their requirements and conditions for supporting student engagement 
mainly in traditional teaching environments and we have confirmed their validity and applicability 
also in MOOCs. This is the first time that such research validation takes place for eCourses and 
more specifically for MOOCs, which is a relatively new field. MOOCs have unique characteristics 
and our research was based on new data to confirm, prioritize and also emphasize some existing 
pedagogical concepts valid in traditional education. Consequently, we have confirmed that specific 
requirements and conditions on feedback and formative assessment for advancing student 
engagement that are applicable in traditional teaching and learning practices, are also applicable 
in MOOCs. Furthermore, we have formulated and ranked fourteen related recommendations as 
good practices based on review comments given by students that completed any of the seven most 
popular courses in a specific online review platform, namely Coursetalk. The ranking was based on 
the frequency of occurrence of the same or similar comments given by the students and we have 
assumed that the higher the number of comments that support one recommendation, the higher is 
the importance of this recommendation. 

In this way, we have concluded some very interesting findings. For example, MOOCs can really 
benefit and encourage students’ activity and consequently their engagement, when assignments 
are based on practical problems with clear expected output that give the sense of completeness to 
the students and are not that easy to address but are sufficiently challenging and interesting. 

Also MOOC students really appreciate it when self and peer assessments deal with more 
comprehensive assignments and are formative as well, i.e. specifically intended to generate feedback 
on performance and improve and accelerate learning rather than just providing a mark. 

On the other hand, some specific practices that are applied in MOOCs, do not seem to be that 
important for the students, for example, use of the same peers in all peer-assessment exercises, 
or the provision of feedback on assignments in time before the next assignment, or no restrictions 
on the number of quiz attempts. 

The results of this research are a first attempt to shed some light on how student engagement in 
MOOCs can be improved via specific formative assessment and feedback practices. However, the 
dataset was rather limited and it was based only on the MOOC participants that provided feedback 
through the Coursetalk platform. Further research could focus on comments and reviews collected 
directly from the MOOC participants in regards to formative and feedback assessment practices 
and how they can affect their engagement in the course. 
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